An L2-boosting algorithm for estimation of a regression function
نویسندگان
چکیده
An L2-boosting algorithm for estimation of a regression function from random design is presented, which consists of fitting repeatedly a function from a fixed nonlinear function space to the residuals of the data by least squares and by defining the estimate as a linear combination of the resulting least squares estimates. Splitting of the sample is used to decide after how many iterations of smoothing of the residuals the algorithm terminates. The rate of convergence of the algorithm is analyzed in case of an unbounded response variable. The method is used to fit a sum of maxima of minima of linear functions to a given data set, and is compared with other nonparametric regression estimates using simulated data.
منابع مشابه
A HYBRID SUPPORT VECTOR REGRESSION WITH ANT COLONY OPTIMIZATION ALGORITHM IN ESTIMATION OF SAFETY FACTOR FOR CIRCULAR FAILURE SLOPE
Slope stability is one of the most complex and essential issues for civil and geotechnical engineers, mainly due to life and high economical losses resulting from these failures. In this paper, a new approach is presented for estimating the Safety Factor (SF) for circular failure slope using hybrid support vector regression (SVR) and Ant Colony Optimization (ACO). The ACO is combined with the S...
متن کاملA Boosting Algorithm for Estimating Generalized Propensity Scores with Continuous Treatments.
In this article, we study the causal inference problem with a continuous treatment variable using propensity score-based methods. For a continuous treatment, the generalized propensity score is defined as the conditional density of the treatment-level given covariates (confounders). The dose-response function is then estimated by inverse probability weighting, where the weights are calculated f...
متن کاملMachine Learning Models for Housing Prices Forecasting using Registration Data
This article has been compiled to identify the best model of housing price forecasting using machine learning methods with maximum accuracy and minimum error. Five important machine learning algorithms are used to predict housing prices, including Nearest Neighbor Regression Algorithm (KNNR), Support Vector Regression Algorithm (SVR), Random Forest Regression Algorithm (RFR), Extreme Gradient B...
متن کاملA Comparative Study of Parameter Estimation Methods for Statistical Natural Language Processing
This paper presents a comparative study of five parameter estimation algorithms on four NLP tasks. Three of the five algorithms are well-known in the computational linguistics community: Maximum Entropy (ME) estimation with L2 regularization, the Averaged Perceptron (AP), and Boosting. We also investigate ME estimation with L1 regularization using a novel optimization algorithm, and BLasso, whi...
متن کاملKernel density classification and boosting: an L2 analysis
Kernel density estimation is a commonly used approach to classification. However, most of the theoretical results for kernel methods apply to estimation per se and not necessarily to classification. In this paper we show that when estimating the difference between two densities, the optimal smoothing parameters are increasing functions of the sample size of the complementary group, and we provi...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- IEEE Trans. Information Theory
دوره 56 شماره
صفحات -
تاریخ انتشار 2010